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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
04/06/2009 has been entered. 

Remarlfs 

2. Receipt of Applicant's Amendment, filed on 04/06/2009, is acknowledged. The 
amendment includes the amending of the specification, the amending of claims 1 , 6-8, 
and 18-19, and the cancellation of claim 2. 

Priority 

3. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 

Specification 

4. The disclosure is objected to because of the following informalities: The 
proposed amendment to the specification filed on 03/05/2009 is unacceptable. 
Specifically, the proposed amendment seeks to amend Paragraph 67. However, the 
proposed amendment merely contains one sentence instead of the required entirety of 
Paragraph 67. The examiner is unclear as to whether the sentence in the proposed 
amendment filed on 03/05/2009 is now entirely Paragraph 67, or is merely a part of 
Paragraph 67. The Examiner suggests that Applicant resubmit the proposed 
Amendment with the entire Paragraph 67 to ensure that the amendment is entered 
correctly. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1,3-19 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Hyon (U.S. PGPUB 2002/0077135) in view of Baker (U.S. Patent 6,546,417), and 
further in view of Ostermann et al. (U.S. Patent 6,990,452). 

7. Regarding claim 1 , Hyon teaches a display device comprising: 

A) a first storage portion for storing beforehand a character code for specifying each of 
said predetermined kind of character (Paragraphs 23, and 41-42, Figure 4); 

B) a second storage portion for storing an image code for specifying said registered 
image and non-standardized image data corresponding to said image code in a 
correlated manner, said registered image being generated by a user (Paragraphs 23, 
25, and 44, Figure 2); 

C) a display output portion for outputting said predetermined kind of character and said 
registered image (Paragraph 21, Figure 2); and 

D) a display control portion for causing said display output portion to output 
corresponding said predetermined kind of character and said registered image both 
belonging to an identical sentence based on display data containing a series of said 
character code, text attribute data, and said image code (Paragraphs 21 and 41-42); 
and 

K) when displavinq a vertical writing (Paragraph 25). 

The examiner notes that Hyon teaches "a first storage portion for storing 
beforehand a character code for specifying each of said predetermined kind of 
character" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23) and "the user can input a text including typical 
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characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41 ). The examiner further notes that 
Hyon teaches "a second storage portion for storing an image code for specifying 
said registered image and non-standardized image data corresponding to said 
image code in a correlated manner, said registered image being generated by a 
user" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that it is clear that the emoticons which are edited, changed, and user-generated 
must be stored in the RAM of storage 18 since items stored in RAM can be written to 
whereas items stored in ROM cannot be written (i.e. are Read-Only). The examiner 
further notes that Hyon teaches "a display output portion for outputting said 
predetermined kind of character and said registered image" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21). The examiner 
further notes that Hyon teaches "a display control portion for causing said display 
output portion to output corresponding said predetermined l(ind of character and 
said registered image both belonging to an identical sentence based on display 
data containing a series of said character code, text attribute data, and said 
image code" as "a display 16 outputs display data and text messages generated in the 
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mobile terminal. An LCD (Liquid Crystal Display) can be used as the display 16" 
(Paragraph 21), "If the user selects "smile" in the display 6 by manipulating a left/right 
directional key, emoticons that belong to the group "smile" are displayed in a display 7. 
When the user chooses one of the "smile" emoticons " . " in the display 7, the selected 
emoticon is stored as part of the short message as shown in a display 8. Then, the user 
can input a text including typical characters, special characters, or emoticons within the 
range of a transmittable SMS message, by changing input modes" (Paragraph 41), and 
"After an intended text is completed, the user instructs the mobile terminal to transmit 
the SMS message including the emoticon... the mobile terminal transmits the stored 
message" (Paragraph 42). The examiner further notes that it is common knowledge 
that when text messages are received via a device, the entire message is displayed at 
once. The examiner further notes that Hyon teaches " when displavinq a vertical 
writing " as "Emoticons are usually divided into two categories: oriental ones used 
among Asians including Koreans and Japanese, and occidental ones mainly used in the 
U.S. and Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal 
ones prevail in the latter case. Therefore, storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
E) said display control portion having image transforming means for transforming said 
registered image to be displayed according to said text attribute data; 

G) wherein said text attribute data contains size attribute data indicating a character 
size of said text; and 

H) said image transforming means that scales up/down said registered image to adjust 
a height of said registered image to a height of said text, serving as a reference in 
accordance with said attribute data; 

I) so that the height of said registered image matches the height of said text 
sandwiching said registered image when displaying a horizontal writing ; 

J) to adjust a width of said registered image to a width of said text, serving as a 
reference, in accordance with said size attribute data : 
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K) so that the width of said registered image matches the width of said text 
sandwiching said registered image . 

Baker, however, teaches "said display control portion having image 
transforming means for transforming said registered image to be displayed 
according to said text attribute data" as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "wherein said text attribute 
data contains size attribute data indicating a character size of said text" as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
scaling the size of the icon graphics are also provided... at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "said 
image transforming means that scales up/down said registered image to adjust a 
height of said registered image to a height of said text, serving as a reference in 
accordance with said attribute data" as "in order to accommodate the use of different 
size fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40), "so that the height of said registered image matches 
the height of said text sandwiching said registered image when displaying a 
horizontal writing " as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40), " to adjust a width of said registered image to a width of 
said text, serving as a reference, in accordance with said size attribute data " as 
"in order to accommodate the use of different size fonts in the mailbox display, means 
for scaling the size of the icon graphics are also provided... at least one image for each 
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icon is stored, tine icon most closely matching the point size of the font is chosen and 
then scaled as needed to better match the font point size" (Column 8, lines 26-40), and 
" so that the width of said registered image matches the width of said text 
sandwiching said registered image " as "in order to accommodate the use of different 
size fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baiter's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Balcer (Column 5, lines 40-55). 

Hyon and Baiter do not explicitly teach: 
F) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined kind of character in a line of 
said identical sentence" as "The sender may also insert emoticons 103 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile, 
"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)"■ As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
100. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 



Application/Control Number: 10/516,881 
Art Unit: 2168 



Page 8 



It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 3, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains color attribute data indicating at least a fore 
color of a corresponding predetermined kind of character; and 

B) said image transforming means converts a color of said registered image according 
to said color attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
color attribute data indicating at least a fore color of a corresponding 
predetermined kind of character" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color" (Column 1 1 , lines 43-50) and "said image transforming means 
converts a color of said registered image according to said color attribute data" 
as "The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color" (Column 1 1 , lines 
43-50). 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 4, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein when said registered is a gray image, said image transforming means 
converts each pixel of said registered image into a color made by mixing the fore color 
and a back color of said text at a ratio according to a pixel value of said predetermined 
kind of character. 

Ostermann, however, teaches "wherein when said registered is a gray 
image, said image transforming means converts each pixel of said registered 
image into a color made by mixing the fore color and a back color of said text at a 
ratio according to a pixel value of said predetermined kind of character" as "The 
increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
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the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 5, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains decoration attribute data indicating a type of 
a decoration applied to a corresponding predetermined kind of character; and 

B) said image transforming means decorates said registered image according to said 
decoration attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
decoration attribute data indicating a type of a decoration applied to a 
corresponding predetermined kind of chiaracter" as "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means... In this case, in a sentence such as "Hi John, :-) are you pleased that the stock 
market is up?" the underlining represents the highlighting wherein the sender chooses 
to begin the smile at the beginning of the word "are" and to continue the smile through 
the word "up". The method comprises receiving the indicated duration of the emoticon 
and presenting the chosen duration of the emoticon" (Column 1 1 , lines 43-61) and 
"said image transforming means decorates said registered image according to 
said decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
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the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 6, Hyon teaches a method comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image being generated by a user and stored in a storage portion 
different from a storage portion of storing the predetermined kind of character 
(Paragraphs 23 and 44); 

D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42); 

J) when displaying a vertical writing (Paragraph 25). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that Hyon teaches "said registered image being generated by a user and 
stored in a storage portion different from a storage portion of storing the 
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predetermined kind of character" as "A storage 18 is comprised of a ROM (Read 
Only Memory) and a RAM (Random Access Memory) for storing programs and data, 
and a voice memory. The storage 18 stores an operation program for inputting 
emoticons and a plurality of emoticons in the form of a bit map according to the 
embodiment of the present invention. The plurality of emoticons are stored by groups in 
the storage 18 to facilitate selection of emoticons" (Paragraph 23) and "In another 
embodiment, the user can add hieroglyphics to basic emoticons stored by a 
manufacturer. The hieroglyphics can be stored in three ways: the user directly stores 
them, receives them from a base station, and the stores them, or downloads them from 
the Internet" (Paragraph 44). The examiner further notes that it is clear that the 
emoticons which are edited, changed, and user-generated must be stored in the RAM 
of storage 1 8 since items stored in RAM can be written to whereas items stored in ROM 
cannot be written (i.e. are Read-Only). The examiner further notes that items which are 
clearly not edited/changed/manipulated (i.e. text characters) would be stored in ROM of 
storage 18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
IHersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "displaying said predetermined kind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that Hyon teaches " when displaying a 
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vertical writing " as "Emoticons are usually divided into two categories: oriental ones 
used among Asians including Koreans and Japanese, and occidental ones mainly used 
in the U.S. and Europe. Vertical hieroglyphics prevail in the former case, whereas 
horizontal ones prevail in the latter case. Therefore, storage 18 stores oriental or 
occidental emoticons according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 

F) wherein said text attribute data contains size attribute data indicating a character 
size of said text; and 

G) said transforming step includes scaling up/down said registered image to adjust a 
height of said registered image to a width of said text, serving as a reference in 
accordance with said attribute data; 

H) so that the height of said registered image matches said height of said text 
sandwiching said registered image when displaving a horizontal writing: 

I) to adjust a width of said registered image to a width of said text, serving as a 
reference, in accordance with said size attribute data : 

J) so that the width of said registered image matches the width of said text sandwiching 
said registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "wherein 
said text attribute data contains size attribute data indicating a character size of 
said text" as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
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image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), "said image transforming means that scales up/down said registered 
image to adjust a height of said registered image to a height of said text, serving 
as a reference in accordance with said attribute data" as "in order to accommodate 
the use of different size fonts in the mailbox display, means for scaling the size of the 
icon graphics are also provided... at least one image for each icon is stored, the icon 
most closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "so that the height of said 
registered image matches the height of said text sandwiching said registered 
image when displaying a horizontal writing " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), " to adjust a width of said 
registered image to a width of said text, serving as a reference, in accordance 
with said size attribute data " as "in order to accommodate the use of different size 
fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40), and " so that the width of said registered image 
matches the width of said text sandwiching said registered image " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
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to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
E) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined kind of character in a line of 
said identical sentence" as "The sender may also insert emoticons 1 03 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile, 
"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)"■ As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
1 00. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 7, Hyon teaches a viewer program comprising: 
A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code (Paragraphs 23, 25, and 44, Figure 2); 
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B) said registered image being generated by a user and stored in a storage portion 
different from a storage portion of storing the predetermined kind of character 
(Paragraphs 23 and 44); 

D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 

42); and 

J) when displaying a vertical writing (Paragraph 25). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that Hyon teaches "said registered image being generated by a user and 
stored in a storage portion different from a storage portion of storing the 
predetermined kind of character" as "A storage 18 is comprised of a ROM (Read 
Only Memory) and a RAM (Random Access Memory) for storing programs and data, 
and a voice memory. The storage 18 stores an operation program for inputting 
emoticons and a plurality of emoticons in the form of a bit map according to the 
embodiment of the present invention. The plurality of emoticons are stored by groups in 
the storage 18 to facilitate selection of emoticons" (Paragraph 23) and "In another 
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embodiment, the user can add hieroglyphics to basic emoticons stored by a 
manufacturer. The hieroglyphics can be stored in three ways: the user directly stores 
them, receives them from a base station, and the stores them, or downloads them from 
the Internet" (Paragraph 44). The examiner further notes that it is clear that the 
emoticons which are edited, changed, and user-generated must be stored in the RAM 
of storage 1 8 since items stored in RAM can be written to whereas items stored in ROM 
cannot be written (i.e. are Read-Only). The examiner further notes that items which are 
clearly not edited/changed/manipulated (i.e. text characters) would be stored in ROM of 
storage 18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
Hersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "displaying said predetermined l<ind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. The 
examiner further notes that Hyon teaches " when displaying a vertical writing " as 
"Emoticons are usually divided into two categories: oriental ones used among Asians 
including Koreans and Japanese, and occidental ones mainly used in the U.S. and 
Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal ones 
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prevail in the latter case. Therefore, storage 18 stores oriental or occidental emoticons 
according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 

F) wherein said text attribute data contains size attribute data indicating a character 
size of said text; and 

G) said transforming step includes scaling up/down said registered image to adjust a 
height of said registered image to a width of said text, serving as a reference in 
accordance with said attribute data; 

H) so that the height of said registered image matches said height of said text 
sandwiching said registered image when displaying a horizontal writing: 

I) to adjust a width of said registered image to a width of said text, serving as a 
reference, in accordance with said size attribute data : 

J) so that the width of said registered image matches the width of said text sandwiching 
said registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "wherein 
said text attribute data contains size attribute data indicating a character size of 
said text" as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), "said image transforming means that scales up/down said registered 
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image to adjust a height of said registered image to a height of said text, serving 
as a reference in accordance with said attribute data" as "in order to accommodate 
the use of different size fonts in the mailbox display, means for scaling the size of the 
icon graphics are also provided. ..at least one image for each icon is stored, the icon 
most closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "so that the height of said 
registered image matches the height of said text sandwiching said registered 
image when displaving a horizontal writing " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), " to adjust a width of said 
registered image to a width of said text, serving as a reference, in accordance 
with said size attribute data " as "in order to accommodate the use of different size 
fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40), and " so that the width of said registered image 
matches the width of said text sandwiching said registered image " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
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E) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined l(ind of character in a line of 
said identical sentence" as "The sender may also insert emoticons 103 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile, 
"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)". As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
100. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 8, Hyon teaches a computer readable recording medium 

comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image being generated by a user and stored in a storage portion 
different from a storage portion of storing the predetermined kind of character 
(Paragraphs 23 and 44); 
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D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42); and 

J) when displaying a vertical writing (Paragraph 25). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that Hyon teaches "said registered image being generated by a user and 
stored in a storage portion different from a storage portion of storing the 
predetermined kind of character" as "A storage 18 is comprised of a ROM (Read 
Only Memory) and a RAM (Random Access Memory) for storing programs and data, 
and a voice memory. The storage 18 stores an operation program for inputting 
emoticons and a plurality of emoticons in the form of a bit map according to the 
embodiment of the present invention. The plurality of emoticons are stored by groups in 
the storage 18 to facilitate selection of emoticons" (Paragraph 23) and "In another 
embodiment, the user can add hieroglyphics to basic emoticons stored by a 
manufacturer. The hieroglyphics can be stored in three ways: the user directly stores 
them, receives them from a base station, and the stores them, or downloads them from 
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the Internet" (Paragraph 44). The examiner further notes that it is clear that the 
emoticons which are edited, changed, and user-generated must be stored in the RAM 
of storage 1 8 since items stored in RAM can be written to whereas items stored in ROM 
cannot be written (i.e. are Read-Only). The examiner further notes that items which are 
clearly not edited/changed/manipulated (I.e. text characters) would be stored in ROM of 
storage 18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
Hersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "displaying said predetermined kind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. The 
examiner further notes that Hyon teaches " when displaying a vertical writing " as 
"Emoticons are usually divided into two categories: oriental ones used among Asians 
including Koreans and Japanese, and occidental ones mainly used in the U.S. and 
Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal ones 
prevail in the latter case. Therefore, storage 18 stores oriental or occidental emoticons 
according to the cultural area of a user" (Paragraph 25). 
Hyon does not explicitly teach: 
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C) with respect to display data containing a series of a cliaracter code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 

F) wherein said text attribute data contains size attribute data indicating a character 

size of said text; and 

G) said transforming step includes scaling up/down said registered image to adjust a 
height of said registered image to a width of said text, serving as a reference in 
accordance with said attribute data; 

H) so that the height of said registered image matches said height of said text 

sandwiching said registered image when displayinq a horizontal writing: 

I) to adjust a width of said registered image to a width of said text, serving as a 
reference, in accordance with said size attribute data : 

J) so that the width of said registered image matches the width of said text sandwiching 
said registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "wherein 
said text attribute data contains size attribute data indicating a character size of 
said text" as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), "said Image transforming means that scales up/down said registered 
image to adjust a height of said registered image to a height of said text, serving 
as a reference in accordance with said attribute data" as "in order to accommodate 
the use of different size fonts in the mailbox display, means for scaling the size of the 
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icon grapliics are also provided... at least one image for each icon is stored, the icon 
most closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "so that the height of said 
registered image matches the height of said text sandwiching said registered 
image when displaying a horizontal writing " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), " to adjust a width of said 
registered image to a width of said text, serving as a reference, in accordance 
with said size attribute data " as "in order to accommodate the use of different size 
fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided. ..at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40), and " so that the width of said registered image 
matches the width of said text sandwiching said registered image " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baiter (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
E) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined kind of character in a line of 
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said identical sentence" as "The sender may also insert emoticons 103 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile, 
"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)". As shown in FIG. 6, the window 80 
includes a sample group of emoticon Icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
100. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 9, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said transforming means includes image decoration means for decorating 
said registered image to provide continuous decoration to both said registered image 
and neighboring predetermined kind of characters adjacent to said registered image 
according to said decoration attribute data. 

Ostermann, however, teaches "wherein said transforming means includes 
image decoration means for decorating said registered image to provide 
continuous decoration to both said registered image and neighboring 
predetermined kind of characters adjacent to said registered image according to 
said decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
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sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 10, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said image decoration means decorates said registered image, such that 
both said registered image and said neighboring predetermined kind of characters show 
the said fore and back colors continuously. 

Ostermann, however, teaches "wherein said image decoration means 
decorates said registered Image, such that both said registered Image and said 
neighboring predetermined kind of characters show the said fore and back colors 
continuously" as The increased intensity of the emoticon may be accomplished by 
changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
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chosen duration of the emoticon as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 1 1 , Hyon and Baker do not explicitly teach a display device 

comprising: 

A) wherein said image decoration means decorates said registered image to draw a 
continuous underline below both said registered image and said neighboring 
predetermined kind of characters. 

Ostermann, however, teaches "wherein said image decoration means 
decorates said registered image to draw a continuous underline below both said 
registered image and said neighboring predetermined kind of characters" as "The 
sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means... In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the 
indicated duration of the emoticon and presenting the chosen duration of the emoticon 
as the animated entity delivers the message. As mentioned above, the highlighting can 
occur through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
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the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 12, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said image decoration means decorates said registered image to draw a 
continuous cancel line on both said registered image and said neighboring 
predetermined kind of characters. 

Ostermann, however, teaches "wherein said image decoration means 
decorates said registered image to draw a continuous cancel line on both said 
registered image and said neighboring predetermined kind of characters" as "The 
sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. ..In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the 
indicated duration of the emoticon and presenting the chosen duration of the emoticon 
as the animated entity delivers the message. As mentioned above, the highlighting can 
occur through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner notes that it is clear that Ostermann's method can use various 
computer font techniques to apply to sentences with emoticons, such as underlines and 
highlighters. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 13, Hyon further teaches a display device comprising: 
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A) wherein the first storage portion is a ROM (Paragraph 23); and 

B) the second storage portion is a RAM (Paragraphs 23 and 44). 

The examiner notes that Hyon teaches "wherein the first storage portion is a 
ROM" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23). The examiner further notes that items which 
are clearly not edited/changed/manipulated (i.e. text characters) would be stored in 
ROM of storage 18. Moreover, the examiner further wishes to state that it is well known 
that text characters from a particular font are stored in ROM and not RAM (see cited 
reference of Hersum (see column 4, lines 55-67 for reference). The examiner further 
notes that Hyon teaches "the second storage portion is a RAIVI" as "A storage 1 8 is 
comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) for 
storing programs and data, and a voice memory. The storage 18 stores an operation 
program for inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups in the storage 18 to facilitate selection of emoticons" (Paragraph 23) 
and "A plurality of emoticons are pre-stored in a mobile terminal so that a user easily 
select an intended emoticon in the present invention. In another embodiment, the user 
can add hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics 
can be stored in three ways: the user directly stores them, receives them from a base 
station and the stores them, or downloads them from the Internet. The user can change 
and edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner 
further notes that it is clear that the emoticons which are edited, changed, and user- 
generated must be stored in the RAM of storage 18 since items stored in RAM can be 
written to whereas items stored in ROM cannot be written (i.e. are Read-Only). 
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Regarding claims 14-17, Hyon and Baker do not explicitly teach a display 
device, display method, viewer program, and computer-readable recording medium 
comprising: 

A) wherein the text containing the predetermined kind of character and the registered 
image is used in a Hyper Text Markup Language document. 

Ostermann, however, teaches "wherein the text containing the 
predetermined Itind of character and the registered image is used in a l-iyper Text 
Markup Language document" as "FIG. 4(a) illustrates a high-bandwidth architecture 
60 associated with the embodiments of the invention. The system 60 delivers a hyper- 
text mark-up language (HTML) page through the Internet 62 (connected to a web 
server, not shown but embodied in the Internet 62) to a client application 64. The HTML 
page (shown by way of example in FIG. 6) enables the sender to create a multi-media 
message. The client application may be, for example, a web browser such as 
Microsoft's Internet Explorer. RTM.. Other client applications include e-mail and instant 
messaging clients. The sender creates the multi-media message using the HTML page" 
(Column 4, lines 53-63). 

The examiner notes that it is clear that Ostermann's method can use various 
computer font techniques to apply to sentences with emoticons, such as underlines and 
highlighters. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 18, Hyon teaches a display device comprising: 

A) a first storage portion for storing beforehand a character code for specifying each of 
said predetermined kind of character (Paragraphs 23, and 41-42, Figure 4); 

B) a second storage portion for storing an image code for specifying said registered 
image and non-standardized image data corresponding to said image code in a 
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correlated manner, said registered image being generated by a user (Paragraphs 23, 
25, and 44, Figure 2); 

C) a display output portion for outputting said predetermined kind of character and said 
registered image (Paragraph 21, Figure 2); and 

D) a display control portion for causing said display output portion to output 
corresponding said predetermined kind of character and said registered image both 
belonging to an identical sentence based on display data containing a series of said 
character code, text attribute data, and said image code (Paragraphs 21 and 41-42); 
and 

J) when displaying a vertical writing (Paragraph 25). 

The examiner notes that Hyon teaches "a first storage portion for storing 
beforehand a character code for specifying each of said predetermined kind of 
character" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23) and "the user can input a text including typical 
characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41 ). The examiner further notes that 
Hyon teaches "a second storage portion for storing an image code for specifying 
said registered image and non-standardized image data corresponding to said 
image code in a correlated manner, said registered image being generated by a 
user" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
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emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that it is clear that the emoticons which are edited, changed, and user-generated 
must be stored in the RAM of storage 18 since items stored in RAM can be written to 
whereas items stored in ROM cannot be written (i.e. are Read-Only). The examiner 
further notes that Hyon teaches "a display output portion for outputting said 
predetermined l(ind of character and said registered image" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21). The examiner 
further notes that Hyon teaches "a display control portion for causing said display 
output portion to output corresponding said predetermined kind of character and 
said registered image both belonging to an identical sentence based on display 
data containing a series of said character code, text attribute data, and said 
image code" as "a display 16 outputs display data and text messages generated in the 
mobile terminal. An LCD (Liquid Crystal Display) can be used as the display 16" 
(Paragraph 21), "If the user selects "smile" in the display 6 by manipulating a left/right 
directional key, emoticons that belong to the group "smile" are displayed in a display 7. 
When the user chooses one of the "smile" emoticons " . " in the display 7, the selected 
emoticon is stored as part of the short message as shown in a display 8. Then, the user 
can input a text including typical characters, special characters, or emoticons within the 
range of a transmittable SMS message, by changing input modes" (Paragraph 41), and 
"After an intended text is completed, the user instructs the mobile terminal to transmit 
the SMS message including the emoticon... the mobile terminal transmits the stored 
message" (Paragraph 42). The examiner further notes that it is common knowledge 
that when text messages are received via a device, the entire message is displayed at 
once. The examiner further notes that Hyon teaches " when displaying a vertical 
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writing " as "Emoticons are usually divided into two categories: oriental ones used 
among Asians including Koreans and Japanese, and occidental ones mainly used in the 
U.S. and Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal 
ones prevail in the latter case. Therefore, storage 18 stores oriental or occidental 
emoticons according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
E) said display control portion having image transforming means for transforming said 
registered image to be displayed according to said text attribute data; 

G) wherein said text attribute data contains size attribute data indicating a character 
size of said text; and 

H) said image transforming means that scales up/down said registered image to adjust 
a size of said registered image to a size of said text in accordance with the direction 
orthogonal to the arrangement of the texts; 

I) so that a height of said registered image matches said a height of said text 
sandwiching said registered image when displaying a horizontal writing : 

J) a width of said registered image matches a width of said text sandwiching said 
registered image . 

Baker, however, teaches "said display control portion having image 
transforming means for transforming said registered image to be displayed 
according to said text attribute data" as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "wherein said text attribute 
data contains size attribute data indicating a character size of said text" as "in 
order to accommodate the use of different size fonts in the mailbox display, means for 
scaling the size of the icon graphics are also provided. ..at least one image for each icon 
is stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "said 
image transforming means that scales up/down said registered image to adjust a 
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size of said registered image to a size of said text in accordance with the 
direction orthogonal to the arrangement of the texts" as "in order to accommodate 
the use of different size fonts in the mailbox display, means for scaling the size of the 
icon graphics are also provided... at least one image for each icon is stored, the icon 
most closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), "so that a height of said 
registered image matches said a height of said text sandwiching said registered 
image when displaying a horizontal writing " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and " a width of said 
registered image matches a width of said text sandwiching said registered 
image " as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
F) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined kind of character in a line of 
said identical sentence" as "The sender may also insert emoticons 1 03 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile. 
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"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)"■ As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
ICQ. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 19, Hyon teaches a method comprising: 

A) storing beforehand a character code for specifying each of said predetermined kind 
of character (Paragraphs 23, and 41-42, Figure 4); 

B) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner, said registered image 
being generated by a user (Paragraphs 23, 25, and 44, Figure 2); 

C) outputting said predetermined kind of character and said registered image 

(Paragraph 21, Figure 2); and 

D) causing said display output portion to output corresponding said predetermined kind 
of character and said registered image both belonging to an identical sentence based 
on display data containing a series of said character code, text attribute data, and said 
image code (Paragraphs 21 and 41-42); and 

J) when disDlaving a vertical writing (Paragraph 25). 
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The examiner notes tliat Hyon teaclies "storing beforehand a character code 
for specifying each of said predetermined kind of character" as "A storage 18 is 
comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) for 
storing programs and data, and a voice memory. The storage 18 stores an operation 
program for inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups in the storage 18 to facilitate selection of emoticons" (Paragraph 23) 
and "the user can input a text including typical characters, special characters, or 
emoticons within the range of a transmittable SMS message, by changing input modes" 
(Paragraph 41 ). The examiner further notes that Hyon teaches "storing an image 
code for specifying said registered image and registered image data 
corresponding to said image code in a correlated manner, said registered image 
being generated by a user" as "A storage 18 is comprised of a ROM (Read Only 
Memory) and a RAM (Random Access Memory) for storing programs and data, and a 
voice memory. The storage 18 stores an operation program for inputting emoticons and 
a plurality of emoticons in the form of a bit map according to the embodiment of the 
present invention. The plurality of emoticons are stored by groups in the storage 18 to 
facilitate selection of emoticons" (Paragraph 23), "storage 18 stores oriental or 
occidental emoticons according to the cultural area of a user" (Paragraph 25), and "A 
plurality of emoticons are pre-stored in a mobile terminal so that a user easily select an 
intended emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that it is clear that the emoticons which are edited, changed, and user-generated 
must be stored in the RAM of storage 18 since items stored in RAM can be written to 
whereas items stored in ROM cannot be written (i.e. are Read-Only). The examiner 
further notes that Hyon teaches "outputting said predetermined kind of character 
and said registered image" as "a display 16 outputs display data and text messages 



Application/Control Number: 10/516,881 
Art Unit: 2168 



Page 37 



generated in the mobile terminal. An LCD (Liquid Crystal Display) can be used as the 
display 16" (Paragraph 21). The examiner further notes that Hyon teaches "causing 
said display output portion to output corresponding said predetermined kind of 
character and said registered image both belonging to an identical sentence 
based on display data containing a series of said character code, text attribute 
data, and said image code" as "a display 16 outputs display data and text messages 
generated in the mobile terminal. An LCD (Liquid Crystal Display) can be used as the 
display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by manipulating a 
left/right directional key, emoticons that belong to the group "smile" are displayed in a 
display 7. When the user chooses one of the "smile" emoticons " . " in the display 7, the 
selected emoticon is stored as part of the short message as shown in a display 8. Then, 
the user can input a text including typical characters, special characters, or emoticons 
within the range of a transmittable SMS message, by changing input modes" 
(Paragraph 41), and "After an intended text is completed, the user instructs the mobile 
terminal to transmit the SMS message including the emoticon... the mobile terminal 
transmits the stored message" (Paragraph 42). The examiner further notes that it is 
common knowledge that when text messages are received via a device, the entire 
message is displayed at once. The examiner further notes that Hyon teaches "when 
displaying a vertical writing " as "Emoticons are usually divided into two categories: 
oriental ones used among Asians including Koreans and Japanese, and occidental 
ones mainly used in the U.S. and Europe. Vertical hieroglyphics prevail in the former 
case, whereas horizontal ones prevail in the latter case. Therefore, storage 18 stores 
oriental or occidental emoticons according to the cultural area of a user" (Paragraph 
25). 

Hyon does not explicitly teach: 
E) said causing step including transforming said registered image to be displayed 
according to said text attribute data 

G) said text attribute data containing size attribute data indicating a character size of 
said text; and 
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H) scaling up/down said registered image to adjust a size of said registered image to a 
size of said text in accordance with the direction orthogonal to the arrangement of the 
texts, in accordance with said size attribute data; 

I) so that a height of said registered image matches said a height of said text 
sandwiching said registered image when displaying a horizontal writing : 

J) a width of said registered image matches a width of said text sandwiching said 
registered image . 

Baker, however, teaches "said causing step including transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "said 
text attribute data containing size attribute data indicating a character size of said 
text" as "in order to accommodate the use of different size fonts in the mailbox display, 
means for scaling the size of the icon graphics are also provided... at least one image 
for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), "scaling up/down said registered image to adjust a size of said registered 
image to a size of said text in accordance with the direction orthogonal to the 
arrangement of the texts, in accordance with said size attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), "so that 
a height of said registered image matches said a height of said text sandwiching 
said registered image when displaying a horizontal writing " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
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scaled as needed to better match the font point size" (Column 8, lines 26-40), and "a 
width of said registered image matches a width of said text sandwiching said 
registered image " as "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baiter's would have allowed IHyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
F) so that said registered image is sandwiched between two parts of said text of said 
predetermined kind of character in a line of said identical sentence. 

Ostermann, however, teaches "so that said registered image is sandwiched 
between two parts of said text of said predetermined kind of character in a line of 
said identical sentence" as "The sender may also insert emoticons 103 into the text of 
the message. The system includes predefined emoticons 96, such as ":-) " for a smile, 
"::-)" for a head nod, "*w*" for an eye wink, and so forth. The predefined emoticons are 
represented either as icons or as text, such as ";-)"■ As shown in FIG. 6, the window 80 
includes a sample group of emoticon icons 96. The sender inserts an emoticon into a 
text message at the location of the cursor 102 by clicking on one of the emoticon icons 
1 00. The sender may also type in the desired emoticon as text. Emoticon icons 96 save 
the sender from needing to type three keys, such as ":" and "-" and ")" for a smile. The 
icons 96 may be either a picture of, say, a winking eye or a icon representation of the 
characters ";-)" 100, or other information indicating to the sender that clicking on that 
emoticon icon will insert the associated emotion 103 into the text at the location of the 
cursor 102" (Column 8, lines 35-50). 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Response to Arguments 
8. Applicant's arguments filed on 03/05/2009 have been fully considered but they 
are not persuasive. 

Applicant argues on pages 14-15 that "Baker only takes horizontal writing 
into account. In contrast, the claimed invention of the present application has 
"image transforming means that scales up/down said registered image to adjust a 
height of said registered image to a height of said text.. .In other words, the 
"image transforming means" of the present invention takes a different approach 
(i.e., height matching v. width matching) based on whether a horizontal writing or 
a vertical writing is displayed". However, the examiner wishes to refer to Column 8 
of Baker which states "in order to accommodate the use of different size fonts in the 
mailbox display, means for scaling the size of the icon graphics are also provided... at 
least one image for each icon is stored, the icon most closely matching the point size of 
the font is chosen and then scaled as needed to better match the font point size" 
(Column 8, lines 26-40). The examiner further wishes to state that because Baker 
scales the icon to a size (i.e. length AND width ) of a respective text font, than as a 
result the claimed heights and widths of the image is adjusted to the heights and widths 
of the claimed text. Specifically, the length and width of any emoticon is always scaled 
to match any requisite text font. The fact that Baker additionally scales another 
dimension nevertheless teaches the claim limitations because the claim limitations 
require scaling the width in one instance and scaling the height in another instance. 
Baker's method always scales both the length and the height, and thus, teaches the 
aforementioned limitations. Moreover, the examiner wishes to state that the primary 
reference of Hyon teaches using both horizontal and vertical writing systems (See 
"Emoticons are usually divided into two categories: oriental ones used among Asians 
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including Koreans and Japanese, and occidental ones mainly used in the U.S. and 
Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal ones 
prevail in the latter case. Therefore, storage 18 stores oriental or occidental emoticons 
according to the cultural area of a user" (Paragraph 25)). The examiner further wise 
Conclusion 

9. The prior art made of record and not relied upon Is considered pertinent to 
applicant's disclosure. 

U.S. Patent 6,987,991 by Nelson on 17 January 2006. The subject matter 
disclosed therein is pertinent to that of claims 1, and 3-19 (e.g., methods automatically 
scale graphics into a text sentence that is sent to another user). 

U.S. PGPUB 2002/0120653 by Kraft et al. on 29 August 2002. The subject 
matter disclosed therein Is pertinent to that of claims 1 , and 3-19 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,584,479 by Chang et al. on 24 June 2003. The subject matter 
disclosed therein is pertinent to that of claims 1, and 3-19 (e.g., methods automatically 
scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,456,305 by Qureshi et al. on 24 September 2002. The subject 
matter disclosed therein is pertinent to that of claims 1 , and 3-19 (e.g., methods 
automatically scale graphics Into a text sentence that Is sent to another user). 

U.S. Patent 5,301 ,106 by Hersum on 05 April 1994. The subject matter 
disclosed therein is pertinent to that of claims 1, and 3-19 (e.g., methods automatically 
scale graphics into a text sentence that is sent to another user). 

Contact Information 

1 0. Any Inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mahesh Dwivedi whose telephone number Is (571) 272- 
2731 . The examiner can normally be reached on Monday to Friday 8:20 am - 4:40 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
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